Strategies to Increase Rigor and Reproducibility of Data in Manuscripts: Reply to Héroux.

REPLY: In a Letter to the Editor, Dr. Martin Héroux presents an analysis of compliance with statistical guidelines (CurranEverett and Benos 2004) in manuscripts published by the Journal of Neurophysiology in 2015. These guidelines (Curran-Everett and Benos 2004) are referenced in the “Information for Authors” for American Physiological Society journals (http://www.the-aps.org/mm/publications/info-for-authors). Héroux focuses on two aspects of the presentation of data: use of standard error of the mean instead of standard deviation, particularly to generate error bars provided in figures, and reporting of exact P values instead of a range (such as P 0.05). As discussed in the statistical guidelines (Curran-Everett and Benos 2004), in most instances standard deviation is a more reliable indicator of variability than standard error, and reliance on a set P can lead to problematic interpretations of data. The American Physiological Society journals have historically treated the statistical guidelines (Curran-Everett and Benos 2004) as a recommendation and not a requirement, which accounts for Héroux’s findings. We are committed to excellence in statistical reporting but must balance this with pragmatic procedures for authors to present their findings. The editors of the journals published by the American Physiological Society are also cognizant of recent reports that research results have often not been reproducible (Landis et al. 2012; Steward and Balice-Gordon 2014; Wadman 2013). A number of proactive steps are underway to improve the rigor and reproducibility of the data reported in the Journal of Neurophysiology. The American Physiological Society’s Publications Committee is currently devising implementation plans for the following recommendations from editors of the Society’s journals: • Description of research methodologies should be improved. Research protocols should be thoroughly described, and adverse events such as changes in animal husbandry conditions should be discussed. The source and catalog/lot numbers of key reagents should be included in the manuscript. • Validation of the selectivity of key reagents should be reported, as well as proof that gene manipulations of model animals are efficacious. • Source data should be made available. As a first step in the implementation plan, the “Instructions for Authors” is being revised, to promote the transparent reporting of research methodology. We are also encouraging authors to include enhanced validation of antibodies and key reagents in their papers. A variety of additional measures will be implemented over the coming years to improve and standardize the reporting of data in the Journal of Neurophysiology and other journals published by the American Physiological Society. While we are committed to keeping pace with other prestigious journals regarding standardizing and improving the reporting of data, we also appreciate that compliance with new requirements requires education of both authors and reviewers. Thus our plan to enhance rigor and reliability will be phased in over time and will be accompanied by a variety of educational activities, as well as the development of tools such as checklists. We are also cognizant that there are disagreements about which experimental details should be reported and how differences in procedures and findings between studies should be evaluated (Morris 2012; Wasserstein and Lazar 2016). Thus our evolving plans will certainly allow flexibility. The journal’s monthly newsletter (http://jnpeditor.org/ Newsletter.html) will provide ongoing information to our readers about the implementation of new standards for reporting of data in the Journal of Neurophysiology.